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Abstract 

We investigate a two-player zero-sum stochastic differential game in which one of the players 
has more information on the game than his opponent. We show how to construct numerical 
schemes for the value function of this game, which is given by the solution of a quasilinear partial 
differential equation with obstacle. 
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1 Introduction 

In 1967 Aumann and Maschler presented their celebrated model for games with incomplete infor- 
mation, see [1] and references therein. The game they consider consists in a set of, say /, standard 
discrete time two person zero-sum games. At the beginning one of these zero-sum games is picked 
at random according to a probability p. The information which game was picked is transmitted to 
Player 1 only, while Player 2 just knows p. It is assumed that both players observe the actions of the 
other one, so Player 2 might infer from the actions of his opponent which game is actually played. It 
turns out that it is optimal for the informed player to play with an additional randomness. Namely 
in a such a way, that he optimally manipulates the beliefs of the uninformed player. 
The extension to two-player zero-sum stochastic differential games has recently been given by Cardalia- 
guet and Rainer in |10j , [5] , where the value function is characterized by the unique viscosity solution 
of a Hamilton Jacobi Isaacs (HJI) equation with an obstacle in the form of a convexity constraint 
in p. The HJI equation without obstacle is the one which is also found to characterize stochastic 
differential games in the classical work of Fleming and Souganidis [T5| . The probability p appears as 
an additional parameter in which the value function has to be convex. 

In Cardaliaguet [S] an approximation scheme for the value function of deterministic differential games 
with incomplete information is introduced. An extension of to deterministic games with information 
incompleteness on both sides is given in the work of Souquiere [22]. We consider the case where the 
underlying dynamic is given by a diffusion with controlled drift but uncontrolled non-degenerate 
volatility. In constrast to [5] and j22j we can work on the problem under a Girsanov transform. 
This transform is a well known tool to consider stochastic games with complete information in the 
context of backward stochastic differential equations (BSDEs) (see Hamadene and Lepeltier [13). An 
approximation of the value function of a stochastic differential game via BSDEs has been discussed 
in Bally [5]. Different to [2] our algorithm is closely related to the work of Barles and Souganidis 
[3] who consider monotone approximation schemes for fully nonlinear second order partial differential 
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equations. The latter was also applied in the recent work of Fahim, Touzi and Warin |12| where fully 
nonlinear parabolic PDEs are treated. As in [12] we use a kind of finite difference scheme for the HIJ 
backwards in time and combine it with taking the convex hull in p at each time step to capture the 
effect of the information incompleteness. Note that this rather direct ansatz using a probabilistic PDE 
scheme also significantly differs from the Makov chain approximation method for stochastic differential 
games described in Kushner [3T]. 

From the very beginning of the investigation of BSDEs initiated by Peng in [53] the close relationship 
with optimal control problems and quasilinear PDEs has been exploited. Consequently, also the 
approximation of solutions to BSDEs and to quasilinear PDEs are closely related. For a survey on 
BSDEs we refer to El Karoui, Peng and Quenez |13j , while a survey on the numerical approximation 
of BSDEs can be found in Bouchard, Elie and Touzi [5]. In this sense our result can also be interpreted 
as approximation of the solutions to the BSDEs which appear in the BSDE representation of the value 
function for stochastic differential games with incomplete information in |17| . 

The outline of the paper is as follows. In section 2 we describe the game and restate the results of [TU] 
and [8] which build the basis for our investigation. In section 3 we present the approximation scheme 
and give some regularity proofs. Section 4 is devoted to the convergence proof. 

2 Setup 

2.1 Formal description of the game 

Let C{[t[)^T]\W^) be the set of continuous functions from R to K.'', which are constant on (— oo,to] 
and on [T, +oo). We denote by Bs{ujb) — wb(s) the coordinate mapping on C{[to,T];M.'^) and define 
H = {Us) as the filtration generated by s Bs- We denote rjj = {w e C([t, T]; R'*)} and Ut^s the 
a- algebra generated by paths up to time s in ^lt. Furthermore we provide C{[to,T];W^) with the 
Wiener measure P" on {T-Ls)- 

In the following we investigate a two-player zero-sum differential game starting at a time t > Iq with 
terminal time T. For any fixed initial data t £ [tQ,T],x 6 R'' the two players control a diffusion on 

{C{[t,T];R^),{nt^,),^[t^T],n,P") given by 

^Xl^x,u,v ^ ^(^^ x*'^'"'", Us,Vs)ds + ais, X*^^ = x. (1) 

where we assume that the controls of the players u, v can only take their values in some compact 
subsets of some finite dimensional spaces, denoted by U, V respectively. 
The aim of the game is to optimize 

(i) running costs: ik)^e{l,...J} ■ [to,T] xR'^xUxV^R 

(ii) terminal payoffs: (gi)ie{i....,/} : R'' — > K, 

which are chosen according to a probability p G A(/) before the game starts. At the beginning of the 
game this information is transmitted only to Player 1. We assume that Player 1 chooses his control 
to minimize. Player 2 chooses his control to maximize the expected payoff. Furthermore we assume 
both players observe their opponents control. So Player 2, knowing only the probability pi for scenario 
i G {I, . . . , /} at the beginning, will try to guess the missing information from the behavior of his 
opponent. 

The following will be the standing assumption throughout the paper. 
Assumption (A) 

(i) b : [to,T] X X U X V M."^ is bounded and continuous in all its variables and Lipschitz 
continuous with respect to (t^x) uniformly in (w, u). 

(ii) For 1 < k,l < d the function ak,i '■ [tQ,T] X R'' ^ R is bounded and Lipschitz continuous with 
respect to {t,x). For any {t,x) £ [0,T] x R*^ the matrix a*{t,x) is non-singular and {a*)~-^{t,x) 
is bounded and Lipschitz continuous with respect to {t,x). 

(iii) : [tQ,T] xW^ xU xl/— >Ris bounded and continuous in all its variables and Lipschitz 
continuous with respect to uniformly in {u,v). {gi)iei ■ R"^' — ;> R is bounded and uniformly 
Lipschitz continuous. 
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(iv) Isaacs condition: for all {t,x,(„p) £ [to,T] x R'* x M"* x A(/) 

(2) 

By assumption (A) the Hamiltonian H is Lipschitz continuous in {S,,p) uniformly in (i, x) and Lipschitz 
continuous in (t, x) with Lipschitz constant c(l + |^|), i.e. it holds for all t^t' G [0,T], x,x' G 

e,C'eM^p,y G A(/) 

|i/(t,x,e,p)| <c(l + |e|) (3) 

and 

\H{t,x,i,p) - H{t', x', < c(i + ~ + + cie - i'\ + c\p^p'\. (4) 

2.2 Strategies and value function 

We now give the necessary definitions and the results of [5] and [TU] on which we will base our 
investigation. 

Definition 2.1. For any t G [to, T[ an admissible control u ~ {us)s^[t.T] for Player 1 is a progressively 
measurable process with respect to the filtration {'Ht,s)s£[t,T] with values in U . The set of admissible 
controls for Player 1 is denoted by U{t). 

The definition for admissible controls v = {vs)se[t,T] for Player 2 is similar. The set of admissible 
controls for Player 2 is denoted by V(t). 

Definition 2.2. A strategy for Player 1 at time t G [to,T[ is a map a : [t,T] x C{[t,T]]R'') x 
L'^{[t,T];V) U which is nonanticipative with delay, i.e. there is (5 > such that for all s G [t,T] 
for any /, /' G C{[t,T];R'^) and <?,<?' G L°{[t,T];V) it holds: / = /' and 5 = g' a.e. on [t,s] 
Q^('i /> 9) = f'l 9') on [t, s + 6]. The set of strategies for Player 1 is denoted by A{t). 
The definition of strategies /3 : [t, T] x C{[t, T]; R'^) x ^^([i, T];U) for Player 2 is similar. The set 
of strategies for Player 2 is denoted by B{t). 

With Definition 2.2. it is possible to prove via a fixed point argument the following Lemma, which is 
a slight modification of Lemma 5.1. in [TU] . 

Lemma 2.3. To each pair of strategies (a,/?) G A{t) x B(t) one can associate a unique couple of 
admissible controls {u, v) G iy({t) x V{t), such that for all ui G C{[t, T]; R'') 

a(s, w, = Ms(cj) and I3{s,uj,u{uj)) = Vs{u!) . 

A characteristic feature of games with incomplete or asymmetric information is that the players have 
to find a balance between acting optimally according to their information and hiding it. To this end 
it turns out that he will give his behavior a certain additional randomness. This effect is captured in 
the following definition. 

Definition 2.4. A random strategy for Player 1 at time t G [to,T[is a a pair ((51q, ^/q, Pq,), a), where 
{na,ga,Pa) is a probability space in Z and a : [t,T] x D.^ x C([t,T];R'') x L°{[t,T];V) U satisfies 

(i) a is a measurable function, where Via is equipped with the cr- field Qa-, 

(u) there exists S > such that for all s G [t,T] and for any /, /' G C{[t,T]]R'^) and g,g' G 
L"{[t,T]]V)) it holds: 

f = f and g = g' a.c. on [t, s] => a{-, f,g) = a{-, f ,g') a.c. on [t, s + 6] for any co G fla- 
The set of random strategies for Player 1 is denoted by A^it). 

The definition of random strategies ((rj/3,a/?,P/3),/3), where /3 : [t,T]xnpxC{[t,T];R'^)xL^{[t,T];U) 
V for Player 2 is similar. The set of random strategies for Player 2 is denoted by B^{t). 
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Remark 2.5. Again one can associate to each couple of random strategies {a, (3) G A^{t) x B^{t) for 
any (cjQ,,a;^) g x fJ^ a unique couple of admissible strategies ^ ^"a."/3) g K(^t"j x V(t), such 

that for all a; e C([t, T]; M'^), s G [t, T] 

a(s,Lj„,Lj,w"-"''(t^)) = and /3(s, w^, w, m"-"'' (w)) = i;^ "'"^^ (w) . 

Furthermore (wq,, w^) — !■ (^""'"(3 ^ jg ^ measurable map, from fta x fi^ equipped with the cr-field 

Ga ® 0/3 to V(t) x U{t) equipped with the Borel cr-field associated to the L^-distance. 

For any {t,x,p) G [to,T[xR'^ x A(/), a G /? G we set 

/ 

J{t,x,p,a,l3) ^ Ea,,^ 

i=l 

where as in Remark 2.5. we associate to aiJ5 for any (wg. jW^) G fig. x 17^ the couple of controls 
The process X*-^'"-'' is then defined for any (wa.,w^) as solution to the SDE (1) 
with the associated controls. Furthermore Ea^.^ is the expectation on fig. x x C{[t,T];M.'^) with 
respect to the probability Pg. P^g (g) P", where P° denotes the Wiener measure on C{[t, T]]R'^). 



{ai)s,l3s)ds + gi{X. 



(5) 



Under assumption (A) the existence of the value of the game and its characterization as a viscosity 
solution to an obstacle problem is shown in [8].|10j. 

Theorem 2.6. For any {t,x,p) G [tQ,T[xM.'^xA{I) the value of the game with incomplete information 
V{t,x,p) is given by 



(6) 



V{t,x,p) = infag(_4.(t))7 sup^gB.(t) J(i,a;,p,a,/3) 
= sup^gg.(j) infag(^.(t))7 J{t,x,p,d,l3). 
Furthermore the function V : [0,T [xM'' X A(/) — >■ K is the unique viscosity solution to 

( dw I ( B^wW 

minj— + -tr(crcr*(t, a;)!)^?/;) +iJ(t,a;,i:):j«;,p),Aniin (p, -^^j | ^ C^) 

with terminal condition w{T,x,p) — J2iPi9ii^) ' where for all p G A(/), A ^ 

Aminb, ^):= mm , ,„ ■ (8) 



and T/^(^i)(^p) denotes the tangent cone to A(/) atp, i.e. T/^(^j)(^p) = Ua>o(^(-^) ~p)/^ ■ 

Remark 2.7. Unlike the standard definition of viscosity solutions (see e.g. [TT]) the subsolution prop- 
erty to (7) is required only on the interior of A(/) while the supersolution property to (7) is required 
on the whole domain A(/) (see [5] and [IH])- This is due to the fact that we actually consider viscosity 
solutions with a state constraint, namely p G A(/) C M^. For more details we refer to [7]. 



3 Approximation of the value function 
3.1 Numerical scheme 

Our approximation scheme of the value function basically amounts to approximate the solution of the 
obstacle problem (7). In order to do so it is convenient to consider the real dynamics of the game (1) 
under a Girsanov transform. This technique - first applied to stochastic differential games by |18| - 
enables us to decouple the forward dynamics (1) from the controls of the players. As in [5] where this 
transformation is applied in the context of numerical approximation for stochastic differential games 
via BSDE we will use the following approximation for the forward dynamics . 
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For L gN we define a partition of [to, T] witli stcpsize r = ^ by 11'^ = {to, ti, . . . ,tL ~ T}. Tlicn for 
all fc = 0, . . . ,i, a; e K'^, p G A(/) let (X*''-^)sG[t,,T] denote the diffusion 



/ cj{r,Xl^-)dBr 

Jtu 



(9) 



Furthermore we define the discrete process (^^'^)rt=fc,...,L as the standard Euler scheme approximation 
for (9) on 



(10) 



We will approximate the value function (6) backwards in time. To do so we set for all x G W^, p G A(/) 

V^tL,x,p) = {p,gix)) (11) 
and we define recursively for fc = L — 1 , . . . , 



V^{tk-i,x,p) = Vexp(E V'^{tk,X^ ^'"^ ,p) + TH{tk-i,x,Zk-iix,p),p)] , 



where z^-iix^p) is given by 



Zk-i{x,p) = -E V^tk,X^~'''',p)ia*)-\tk-i,x)AB 



(12) 



(13) 



and Vexp denotes the convex hull, i.e. the largest function that is convex in the variable p and does 
not exceed the given function. 



3.2 Some regularity properties 
3.2.1 Monotonicity 

First we show that our scheme fulfills a monotonicity condition which corresponds to the one in [3] 
(2.2). It is well known that this criteria is crucial for the convergence of general finite difference 
schemes. 



Lemma 3.1. Let 



be a uniformly Lipschitz continuous function with Lipschitz constant 



M . Then there exists for all x, a;' G M a 



l"^ with \6\< M 



{x)-(t>{x') = {e,x-x') 



Proof: For </> G the result follows from partial integration with 9 = Dx(j){x + r{x' — x))dr. 
For the case of general Lipschitz continuous function (j) one chooses a sequence of functions (0'^)e>o 
which converges uniformly to 0. Since (p is uniformly Lipschitz continuous, we may assume that the 
absolute value of Dx<j)'^ and hence the corresponding 9'^ are uniformly bounded by the constant M . 
Consequently, possibly passing though a subsequence, there exists a 6* G M'' with \9\ < M such that 
the lemma holds. □ 

With the help of Lemma 3.1 we now establish: 



Lemma 3.2. Lei fc G {0, . . . , i - 1} and 0, : R*^ 

for any cc G M, p G A(/) 



he two Lipschitz continuous functions. Then 



E 



rk.X 



> E 



Ti?(tfc,a:,-E 

T 



cl^{Xlf,){a*)-'{t,,x)/^B'^ ,p) 



^(^fc+i)] +THitk,x,-E 

J T 



^(X^V^J(cT*)-^(t,,x)AB'^ ,p)-TOir), 



where 0(t) is independent of p. 
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Proof: By (4) H is uniformly Lipschitz continuous in ^. So by Lemma 3.1. there exists a. 9 & M."^ 
with \6\ < Af , where M denotes the Lipschitz constant of H, such that 



E 



1 



Hitk,x,-E i;ix'^'_^^)ia*y\tk,x)AB^ ,p)) 



T 



yk.x 



= E 

= E 
= E 



tB, ( ^E 



61, E 



<t>{xlf,)i^*)-\t,,x)AB' 



Ie 

r 



^){Xl':,){l + {B,{a*)-\U,x)AB'^)) 



Since < 4'{x) — ipi^) ^ c for any a; £ R, we have 



E 



> 
> 



^)(X,^;^')(l + (0,(a*)-i(t..,x)Ai?'=)) 



-CE 



|ABfc|>- 



J AS* 



with C := \\Ma ^\\oo independent of {tk,x,p) and r. Furthermore we can explicitely calculate 

1 r°° . . , 1 1 _ 1 



E 



1, 



where F denotes the gamma function 



\x\e dx = , 
(2^)S(r)Wi.|>i' 2f-iF(f) 



r2e 2c2t. 



□ 



3.2.2 Lipschitz continuity in a; 

To show that the Lipschitz continuity in x is preserved under the scheme, we establish the following 
Lemma. 

Lemma 3.3. Let k G {0, . . . , L — 1} and (f> : — >■ R &e a uniformly Lipschitz continuous function 
with Lipschitz constant M . Then for any k G {0, . . . , L — 1}, x,x' G R, p G A(/) 



<t>{.Xl'+i)\ +rH{tk,x,-] 



cl^{xtZ){'J*)-\t,,x)AB'^ ,p) 



-E [cb{Xt+i)] - rH{tk,x', ^E [cl>{x',f,){a*)-Htk,x')AB''] ,p) 

< C"'^\x-x'l 
where C^^'"^ — A/(l + ct) + ct with c independent of p. 
Proof: We fix A: G {0, . . . , L - 1}, x, x' G R, p G A(/) and write 

'_cb{X^f,)~^{x',f,)] +r(^H{tu,x, iE [cp{X^^^,){a*)-\t,,x)AB^] ,p) 

~H{tk,x'4E U(x'^f,){a*)-\t,,x')AB^] ,p) 



E 



= E 



1 



'rk.x' \ 



+T(^F(t,,x,iE [cj){x'^::^^)ia*)-^itk,x)AB'^\ ,p) 

-H{tk,x, iE [cl,{xt:^[){<j*)~Htk,x')AB'^] ,p) 
-r(^i/(tfc,x,iE [l,{xl-^[){a*r\tk,x')AB^] ,p) 

-H{tk,x\^E U{X^^f^){a*)-\tk,x')AB'^] ,p) 



(14) 
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Assume that (p ^ with \Dx<l>\ < M. Fh'st we consider the last term of (14). We have for 9^ 
/o D^(j){x' + ra{tk,x')AB'')dr that le^l < M and 



-E 



Hxt+i)i<^*r\tk,x')AB' 



1 |_ 
r 

< M. 



[cbix')ia*)-\t,,x')AB'^ + e^lAB'^p] | 



Since by (4) the Hamiltonian H is uniformly Lipschitz continuous in a; with Lipschitz constant c(l + 1^|) 
it holds 



T(^H{t,,x, Ie [<l>ix',-^l){a*)-\tk,x')AB''] ,p) - H{t,,x', [q)ix'^'^[)ia*)-\tk, x')AB'^' 
<Tc{l + M)\x-x'\. 



For the remaining terms in (14) we note that by (4) the Hamiltonian H is uniformly Lipschitz con- 
tinuous. So there exists as in Lemma 3.1. a. 9^ 6 M'* with 16*^1 < c, such that 



+T(H{tk,x, -E U{X^,-:^,){a*)-\tk,x)AB'^] ,p) - H{tk,x, -E U{Xl^J^[){a*r\tk,x')AB^ ,p) 



= E 



= E 



c^ixtf,)-Hx;:':,) 



k.x 



E 



cl>{X',f,)(<j*)-Hh,x)AB' -E <t>{xl^[){a*)-\tk.x')AB^ )) 



iHXt:^) - 0(%^i))(l + {0\ {a*)-\tk,x)AB'^)) 



(15) 



-E 



^{Xtfi)iial-'itk.x) - {a*)-'{tk,x'))AB'') 



For the first term of (15) we have with 0-^ Dx(l){X^'^-^ + r{X^'^-^ — X^'^-^))d'. 



(Hxtt^) - 0(^:+" ))(1 + {e\ {<r*)-\U, x)AB'^)) 



?k.x 



E 



= E [{@\X',:, - X^:^,) (1 + {9\ ia*)-\h,x)AB'))_ 

< E [(e^, (1 + {e\ {a*r\tk,x)AB'')){x - x') + {a{t,,x) - a(tfc, x'))Ai?")] 
+ct\x — x'\. 

We finally use Cauchy-Schwartz (note that in the expansion of the square the AB'' parts vanish when 
taking expectation), |0^| < M and the Lipschitz contiunity of a to get 

E [(e^, (1 + {e\ {a*)-\tk, x)AB'')){x - x') + {a{tk,x) - a{tk, x'))AB'')] 



< ME 



((1 + {e\ {a*r\tk, x)AB^)){x - x') + {a{tu,x) - a{tu,x'))AB^y 



< M\x - x'\ (E [l + c|AB'=P]) =^ = M\x - x'\{l + ct)^ < M\x ~ x'\{l + -r). 

For the second term of (15) we use the uniform Lipschitz continuity of (a*)^^ (by assumption (A)) to 
have with the R'^-valued random variable 8'^ := Dx(f>{X^':^^ + r{X^'^^ — x'))dr 

l\cp{X^^^[){{a*r\u,x) - {a*)-\U,x'))AB^)_ 

9\<p{x',f,){{a*r\t,,x) - {a*)-\tk,x'))AB'^) 

= E [{e\ (</>(x') + {Q\ait,x')AB'^Ma*r\tk,x) - {a*r\tk, x'))AB'^)] 
< cMt\x -x'\. 

The case of Lipschitz continuous (p follows by approximation with a sequence of functions (0'^)e>o 
which converges uniformly to (f>. Since (j) is uniformly Lipschitz continuous with constant M, we may 
assume that \Dx(f>'^\ < M for all e > 0. □ 
With the previous Lemma it is easy to show the Lipschitz continuity of V'^{t.,x,p) in x. 
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Proposition 3.4. y^(i.,a;,p) is uniformly Lipschitz continuous in x with a Lipschitz constant that 
depends only on the constants of assumption (A). 



Proof: We will show Proposition 3.5. by induction. With (A) we have that V'^{tL,x,p) is Lipschitz 
continuous in x with a constant that depends only on the constants of assumption (A). Let Mk 
be the Lipschitz constant for {tk, -jp) then by (12) and Lemma 3.3. and since Vex is monotonic, 
we have 

\V^itk-i,x,p) ~V^t,,^i,x',p)\ < A'h{{l + ct)^ + ct) + ct)\x ~ x'\. 

Hence Mk-i ■= Mk{l + ct) + cr is a Lipschitz constant for V'^{tk-i,-,p) and M := M^Ce'-^^ 
for a C independent of T,x,p is a constant dominating the recursively defined Lipschitz constants 

(Mfc)fe=0,...,L. □ 

With the uniform Lipschitz continuity of in x it follows that the value function is uniformly 
bounded. 

Proposition 3.5. {t.,x,p) is uniformly bounded by a constant only depending on the constants of 
assumption (A). 

Proof: Fix k e {0, i - 1}, x G R^, p G A (/). Assume first that is at tk+i continuously differen- 
tiable in the second variable with I I <M. Thenwith9:= DxV'^{tk+i,x+ra{tk,x)AB''',p)dr 



\zkix,p)\ 



= i |E [V-itk+i,x + aitk,x)AB'',p){a*)-\tk,x)AB''] 
= i |E [V^tk+i,x,p){a*)-^{tk,x)AB'^ + OlAB^f] \ 
< M. 



(16) 



Since is by Lemma 3.3. uniformly Lipschitz continuous in x one has (16) in the general case again 
by regularization. 

By (A) V'^{tL,x,p) is bounded by a constant Ml that depends only on the constants of assumption 
(A). Let Mk be a bound for \V'^{tk, -jp)] then by (3) the definition (12) and (16) we have 



E 



V^tk,X^ ^''\p) +TH{tk^i,x,Zk-i{x,p),p)<Mk + cT{l + M) 



□ 



and Ml + cT{l + M) is a constant dominating the recursively defined constants {Mk)k=o,...,L- 
3.2.3 Lipschitz continuity in p 

The Lipschitz continuity of V^lt., x,p) in p can be shown with similar methods. 

Lemma 3.6. Let k G {0, . . .,L ~ 1} and (f> : Mf^ x A(/) — > M &e a uniformly Lipschitz continuous 
function with Lipschitz constant M . Then for any k G {0, . . . ,L — 1}, x G M'', p,p' G A(/) 



E 



,P) 



0(X,^fi,p)J +TH{tk,x,-E y,ix','_^,,p){a*)-\h.x)AB'' 

-E [0(X,^+"i,y)] -rHitk,x,^E [^ix'^-:^,,p')ia*)-\tk,x)AB'^] ,p') 
<C^''^\p-p'l 
where C'^'^ = M{1 + ct) + ct. 

Proof: We fix /c G {0, . . . , L - 1}, a; G R"*, p,p' G A(/). First note that by (4) the Hamiltonian is 
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uniformly Lipschitz in p. Hence 



yk.x 



-Hitk,x,-E c^iX','_^„p')ia*)-Htk,x)AB'^ ,p') 

T L ' J 



E 



< E - +T\^H{t,,x,-E [cj,{X',:^„p){<j*)-\h,x)AB'\ ,p) 

^H{tk,x,^E [cj,{x',f,,p')ia*r\h,x)AB'^] +ct\p~p'\. 

By (4) the Hamiltonian H is uniformly Lipschitz continuous in ^ with a constant c. So by Lemma 
3.L 



E 



1, 



0(X,^;^i,p) - HK+i^P')\ +ri^H{tk,x, -E [^{X'^-:^,,p)ia*)-\tk,x)AB'\ ,p) 

k,x f\/ * \ — 1 / 



H{tk,x,-E c^{Xl'^,,p'){a*)-'{tk.x)AB^ ,p) 



= E 



E 



Hxt:,\,p)-Hx',:,\,p') 



9.E 



<t>{Xl^^,p){a*r\tk,x)AB'' -E ^{Xl^J^„p'){a*r\u,x)AB 



{^{Xt+i^p) - ^{Xtti^p')){l + {9, {a*)-\t,,x)AB'')) 



Assume for now that (j) is differentiable in p with \Dp(t)\ < M. Then with 6 /(, Dp(j){X'^'_^^,p + r{p~ 
p'))dr we have 



E 



i<t>{X',':„p) - <t>iX',':„p')){l + {0, ia*)-\t,,x)AB')) 
E [(e, (1 + {9, x)AB'^)){p - p'))] 



< M\p-p'\ (E [1 + c|AS'^'p])^ = M\p-p'\{1 + ct)^ < M\p-p'\{l + |r), 

where for the first estimate in the last line we used again Cauchy Schwartz as in the previous Lemma. 
The general case follows again by regularization. 

□ 

It is now easy to show the Lipschitz continuity of {t.,x,p) in p as in Proposition 3.4. 

Proposition 3.7. V'^{t.,x,p) is uniformly Lipschitz continuous in p with a Lipschitz constant only 
depending on the constants of assumption (A). 



3.2.4 Holder continuity in t 

Finally we use the Lipschitz continuity of in x to establish the Holder continuity in time. 

Proposition 3.8. For all L E x E R'^ , p E A(/) it holds that {t,,x,p) V^{t,,x,p) is Holder 
continuous in t., in the sense that for all k E {1, . . . , L — 1}, / E {1, . . . L — k}, there exists a constant 
c only depending on the constants of assumption (A), such that 

IVitk+^x^p) - V''{tk,x,p)\ < c\tk+i - tk\^. 
Proof: We fix {x,p) eW^ x Ail). By (12), (3) and the convexity of V in p we have 
\V^tk+i,x,p)~V^itk,x,p)\ 

= V'^{tk+i,x,p) ~Ycxp(e V^{tk+i,X^':^-^^,p) + TH{tk,x,Zk{x,p),p) 

< E\v^tk+ux,p)-V^tk+i,xt+i^p)\ +ct{1 + M), 
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where we used that by (16) \zk{x,p)\ is bounded uniformly in p G by the Lipschitz constant of 

in X. Note that by definition (12) 

{tk+i , Xl'^^ , p) = Vexp (^E \V^ {tk+2 , ^,'+2 , P)] + rH{r, x' , z^+i {x' ,p),p) 

Hence by (A) and the fact that is convex in p we have 
V^{tk+ux,p) ^¥.\v^{tk+uXll^^,p) 
= V^(tk+i,x,p) 

\^^(tA;+2,X^^2^^ ,p) +TH{r,x\zk+i{x',p),p) 

+ ct(1 + M) 



Vcxp I E 



< 



V^tk+i,x,p)~E 
= V^tk+i,x,p)~E[v^tk+2,X^+2^p)\ +cr(l + M). 
Since It = \tk+i — tk\ repeating this now / — 2 times gives 

\V^h+i,x,p)-V'itk,x,p)\ < \v^tk+i,x,p)-E[v^{tk+i,X^-:^i,p) + c{l + M)\tk+i - h\. 
Furthermore by the Lipschitz continutity of in x and (A) it holds 



V^tk+i,x,p)-E V^h+i,X^^-:^^,p) 



< ME 



\xt'+i-x\ < c\tk+i - tk\^ , 



hence 



\V^itk+i,x,p)~V^tk,x,p)\ < M\tk+i~tk\^ +c{l + M)\tk+i-th\. 



□ 



4 Convergence 

Theorem 4.1. Under (A ) we have uniform convergence on the compact subsets of [0, T] x R'' x A(/), 



lim V^{tk,x',p')=V{t,x,p). 



(17) 



Note that by Proposition 3.5. the family {V^ ,t > 0) is uniformly bounded. Furthermore by Proposi- 
tion 3.4., 3.7. and 3.8. the family {V^ > 0) is equicontinuous, hence by Arzela Ascoli compact for 
the topology of uniform convergence. Furthermore any candidate for the limit of as r J, is as a 
limit of convex functions convex in p. 

Let w : [0, T] x R'' x A(/) — >■ R be a candidate for the limit. We will show that w is a viscosity solu- 
tion to (7). Since this property uniquely characterizes the value function V the convergence follows 
immediately. 

4.1 One step a posteriori martingales and DPP 

By construction there exists at each time step t^ for any x G R"* and j» G A(/) a linear combination 
of 7r''''^(x,p), . . . , Tr'^''{x,p) G A(/) such that 



J2 {x, P)^'^' {x,p)^p ^ Af (x, p) = 1 



(18) 
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and 



with 



(19) 
(20) 



where we can choose (x,p) — > A'^(x,p) £ A(/) and (x,p) — > 7r'^(x,p) G Borel measurable. 



Definition 4.2. For alH G /, fc = 0, . . . , L, x G R" and p G A(/) we define the one step feedbacks 
Pfc+f A(/)-valuGd random variables which arc independent of a{Bs) ses.-, such that 

(i) for fc = 0, . . . , L - 1 

(a) if Pi set p*j.'^f = p 

(b) if Pi > 0: p^lf G {■k'''^{x,p), . . . , 7r'='-^(x,p)} with probabihty 



P [Pfe+f = 7r''''(x,p)|(pf=' )je{i,...,i},x'm,p'eAiM{h-M 

(ii) for fc = L set p^'^'^' = e\ 

Furthermore we define one step a posteriori martingales p^'^i = P^'+f , where the index i is a ran- 

dom variable with law p, independent of cr(i3s)se[o,T] and (pf^ )je{i,...,/},a:'6R,p'eA/,;e{i....,L}- The 
martingale property is a direct consequence of the proof of the Lemma given below. 

The following one step dynamic programming is a direct consequence of Definition 4.2. 
Lemma 4.3. For all k = 0, . . . , L - 1, x G R'^, p G A(/) we have 

F^(tfc,x,p) =E V''{tk+i,X^'_^^,plf^)+TH{tk,x,Zk{x,pl'lj^),pl'l^) 

with 



Af(x,p)-^"''("'^))^ 



(21) 



1, 



zkix,pl'l,) = -E V^tk+uX!^:^,,p){a*)-\U,x)AB 



(22) 



Proof: Assume (p)i > for alH = 1, . . . , /. By the construction for all suitable functions / : A(/) 
R it holds 



E[/(p^fi)] = [l{i=.}/(pltf )] - IE [/(Pl'^f 

i=l 4=1 



^p.^Af(x,p)^^^^:^^/((7r'^-^'(x,p))) 



i=l 1=1 
I 



^Af(.x,p)/(^'^^'(x,p)) 



1=1 



and the Lemma follows with (19). 



□ 



11 



4.2 Viscosity solution property 
4.2.1 Viscosity subsolution property of w 

Proposition 4.4. w is a viscosity subsolution of (7) on [0,T] x R'* x Int(A(/)). 

Proof: Let : [0, T] x M x A(/) — > K be a test function such that w ~ (f) has a strict global maximum 
at (i, where p G Int(A(/)). We have to show, that 

min\^^ + Ui{aa*{t,x)Dl4))+H{t,x,D,ct>,p),\^i^ } ^^^^ 

holds at (i, As a limit of convex functions w is convex in p and we have since p G Int(A(J)) 



5^' 



Amin ( P, —{t,X,p) ) > 0. 



So it remains to show 



^ + hT{aa*{t,x)Dl^) + Hit,x,D,4>,p) >0. (24) 

Note that by standard arguments (e.g. [3]) there exists a sequence {tk,Xk,Pk)keN such that ffe = 
Zfc^ = IkT G n"^ converges to t and {xk,Pk) converge to {x,p) and such that — has a global 
maximum at {ik,Xk,Pk)- 

Define 0"^ = + (V^'^(ifc, x^jp^) — (j>{ik,Xk,Pk)) = + A,-. Hence for all x eR,p e A(/) 

l^'^(ife + T,X,p) - (j/{tk + T,X,p) < V'^{tk,Xk,Pk) - 4>'^{tk,Xk,Pk) = 0. 

Set 

Xk+i = Xk+(j{tk,Xk)^B^'' 

and 

T 

By the definition of (12) it holds 

= Yeyip(^[V''{ik + T,Xk+i,Pk) + TH{tk,Xk,Zk,Pk)ds]) -V''{tk,Xk,Pk) 
< ¥.[V^{ik+T,Xk+i,Pk)\ +TH{tk,Xk,Zk,Pk) -V{tk,Xk,Pk)- 

Hence by the monotonicity Lemma 3.2. we have for all t > 

1 



< E 



V^{ik + T,Xk+i,Pk) + TH{tk,Xk, -E [V^{tk + T,Xk+i,Pk){a*y\ik,Xk)l^B'-] ,pk) 
-Viik.Xk^Pk) 
< E _ 

-0^(ffe,Xfc,pfc) + TO(r) 



^^{tk+T,Xk+uPk) + TH{tk,Xk.'^'& [c^^tk + T,Xk+i,pk){<J*)-\ik,Xk)/^B'''] ,Pk) 



By expansion of the smooth function 0"^ we have since 0'^ is equal to with the linear shift A^ the 
inequality (24). □ 

4.2.2 Viscosity supersolution property of w 

Proposition 4.5. w is a viscosity supersolution of (7) on [0,T] x R'' x A(/). 
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Proof: To show that w{t,x,p) is a viscosity supersolution of (7) let </> : [0,r] x M x A(/) be a test 
function, such that w — (j) has a strict global minimum at (t,x,p) with w^t^x^p) — (j){t,x,p) = and 
such that its derivatives are uniformly Lipschitz continuous in p. 
We have to show, that 

min|^ + itr(aa^(<,a;)I?20) + 6(t,x)Z?,0 + Jf(t,.T,D,0,p),A„,in | <0 (25) 

holds at (t,x,p). Observe that, if Amin (^f^) < at {t,x,p), then (25) follows immediately. So we 
assume now Ami,, (|^) > 0- 

By standard arguments (e.g. [3]) there exists a sequence (ik, Xk,Pk)keN such that ~ I^t G IV 
converges to t and (xk^pk) converge to {x,p) and such that — cf) has a global minimum at {ik, Xk,Pk)- 
Define <jf ~ tj) + {V^ {ik, Xk,Pk) ~ <i>^kT^k,Pk)) = + A^. Since the minimum is global, we have 



V^{tk + T,X,p) - 4>^{tk + T,X,p) > V''{tk,Xk,Pk) ~ (l)(tk,Xk,Pk) = 0. 



Note that by the assumption Amin (^f^ j > there exists (5, 77 > such that for all k great enough we 
have 

{-Q;^{t,x,p)z,z) > A5\z\^ \f{x,p) £ Bn{xk,Pk), <e[ife,tfe + r], zeTA(i)(p^). (26) 

Since cf)'^ is a test function for a purely local viscosity notion, one can modify it outside a neighborhood 
of {tk,Xk,Pk), such that for all (s, x) G [t/c, T] xM"^ the function ^'^(s, x, ■) is convex on the whole convex 
domain A(/). Thus for any p e A(/) it holds 

V^is,x,p) > 4'^s,x,p) > 4>^{s,x,pk) + {^is,x,p),p - pk). (27) 

We proceed in several steps. 

(1) First we show a local estimate which is stronger than (27) using (26). 

(2) In the second step we establish estimates for Pfc+i := pf^^'+i' where pf^^^i is defined as one step 
martingale with initial data (tk, Xk,Pk) as in Definition 4.2. 

(3) Then we use the estimates of the second step together with the monotonicity in Lemma 3.3. to 
conclude the viscosity supersolution property. 

Step 1: We claim that there exist r],5 > 0, such that for all r > small enough (meaning k great 
enough) it holds 

V^{tk+T,X,p) > <j)''{tk+T,X,Pk) + {^itk+T,x,pk),p-Pk) +S\p-Pk\^. (28) 

for all X G Bri{xk), p £ A(/). By Taylor expansion in p 

(j)-'it,x,p) > (b^t^x^pk) + {^it,x,p),p - pk) + 25\p - pkl"" (29) 

holds for {x,p) E B,j{xk,Pk), t G [ik,ik + t]. Hence (28) is true locally in p. To establish (28) for all 
p G A(/) we set for p G A(/) \ Int(_B^(pfe)) 

~ - , P-Pk 

P^Pk + -. ^V- 

\P~Pk\ 

So by the convexity of V"^ in p and (29) we have for a p G dV'^~ {ik, Xk,p) 
V^{ik,xk,p) > V^{ik,Xk,p) + {p,p-p) 

> (j)^{tk,Xk,Pk) + {-Q;^{tk,Xk,Pk),P - Pk) + "^Srf + {p,p-p) 

> (j)''{tk,Xk,Pk) + {^^{tk,Xk,Pk),P - Pk) + '^Si]^ + {p —{tk,Xk,Pk),p-p)- 

op op 
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Since ^g^{tk, Xk,Pk) G dV^ {tk,Xk,Pk) and p — p = c{p — pk) (c > 0) and is convex in p it holds 



{P - -g;^[tk,Xk,Pk),p-p) > 0. 
So we have for all p e A(/) \ Int(i3,,(pfe)) 

V^{tk,Xk,p) > (j)'^{tk,Xk,Pk) + {-^[tk,Xk,Pk),P - Pk) + 2(5?7^ 

op 

which gives in the limit for all p e A(/) \ Int(B^(p)) 

wit,x,p) > (f>{i,x,p) + {—{t,x,p),p-p) + 25if. 



(30) 



(31) 



Assume now that (28) does not hold for ape A(/). Hence there exists a sequence {T,Xk^,Pkn) ~^ 



(0, 0,p) with T = ^, pfc„ e A(/) \ B^iPkJ, such that 



^'^(*fc„ + 2;fc„ + Xfc„ , pk„ ) 



< 'i>'^{tk„ +T,Xk„ +Xk„,Pk„) + {^t—{tk„ +T,Xk„ + Xk„,Pk,J,Pk„ ~Pk„) +S\pk„ -Pfc„P 

op 



Thus for n cx), p e A(/) \ Int(i3^(p)) and 



'jit,x,p) < (f>{t,x,p) + {—{t,x,p),p-p) +5rf 



(32) 



which contradicts (31). 
In the following we denote 

Xk+l = Xk+CT{tk.Xk)l^B^K 

where AB'^- = Bi^+r - Bt^. With the estimate (28) we have for r small enough for all p € A{I) 
E[V^tk+T,Xk+up)] 



= E 

> E 

= E 



V^ik + T,Xk+uP)l\x,^,- 



+ l-a:fc|<»? 



-E 



V^tk+T, Xk+i,p)l\x^^^_s,\>v 



(^(l)'^{tk+T,Xk+l,Pk) + ^^^^^^h +T,Xk+l,Pk),P-Pk) +S\p-Pk\'^^ l|Xfc + i-Sj3|<r, 



-E 



0^(tfc +r, Xk+i,p)lix,+,-s,i>^ 



d - - 

(j)''{tk +T, Xk+l,Pk) + (g^'I'^'i^k +T, Xk+l,Pk),p-Pk) + Slix^+i-Xk\<v\P ~ Pk\'^) 



-E 



l|X, + l-Sfc|>T,(^'/'^(ifc +T, Xk+l,p) - (t)^{tk+T,Xk+l,Pk) 



d - - _ 

-g^<l>^{ik +T, Xk+l,Pk),p- Pk) 



Recalling that (f)^ is convex with respect to p, we get for all p € A(/) 

IE[y"(tfc+T,Xfc+l,p)] > E 0^(tfc+T,Xfc+l,Pfe) + (|j0"(tfe+T,Xfc+l,Pfc),p-pfc) 

+5l\x,+^-x,\<^\p - Pk?) 



(33) 



Step 2: Next we establish an estimate for Pfc+i := where V^'^+i is defined as one step martingale 

as in Definition 4.2. with initial data (tk,Xk,Pk)- 

Note that by the one step dynamic programming (21) it holds 



V''{tk,Xk,Pk) = E [V^ {tk + T, Xk+i,Pk+i) + TH{tk,Xk,Zk{xk,Pk+i),Pk+i)] 



(34) 
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Together with V'^{tk, Xk,Pk) = 0^(^fe, Xk,Pk) and the estimate (33) we have for all small enough t > 



i>''itk +T, Xk+i,Pk) + TH{tk, Xk, Zkixk,Pk+i),Pk+i) 



Since Pk+i and AB'*-' arc independent, 0"^ has bounded derivatives and Pk+i is a one step martingale, 
it holds 



E 



d - - _ 
(^(^^(ifc +T,Xk+i,Pk),Pk+i -Pk) 



E 



{^(t>''{ik + T,Xk +c^(^fc,Sfc)AB'^pfc),pfc+l -pk) 
op 



Furthermore by the Markovian inequality and assumption (A) we have 



E 



l|Xfc + i-.Tfc|<,,|Pfc+l -Pk\ 



T(tfc, Sfc)AS'fc|<r,|Pfe+l ^Pk\ 



>c(1-t5)E [|pfe+i 



with a sufficiently small constant c independent of k. Thus 

> K (lf{ik + T,Xk+i,Pk) - (l>^{tk,Xk,Pk) +TH{tk,Xk,Zk{xk,Pk+i),Pk+i) 
+c6{\ - T^)\pk+l -Pk\ 
Since cf)^ has bounded derivatives it holds by assumption (A) 

\&[(l/{tk+T,Xk+l,Pk) - (l>^{tk,Xk,Pk)\\ < CT 

and since E [|zfe(xfe, pfc+i)|] < c (16) it holds by (A) and Holder 

E[TH{tk,Xk,ZkiXk,Pk+l),Pk+l)] < CT. 

Combining (35)- (37) we have for small enough r > and a generic constant c' > 

E[\pk+i - Pk\'] < , r, 

Cd[l — T2 j 

hence for small enough r and a constant c" > 

E[|p;.+i -Pfel'] <c"r. 



(35) 



(36) 
(37) 



(38) 



Step 3: 

Furthermore we have with (35) and the monotonicity Lemma 3.3., since (tk, Xk,Pk) = 4''^ {tk, Xk,Pk) 



0^(tfc +T,Xk+l,Pk+l) - 4>'^(tk,Xk,Pk) +TH{tk,Xk,Zkixk,Pk+l),Pk+l) 



0(r), (39) 



> E 
where 

Zk{xk,Pk+i) = -E [(j)^tk + T,Xk+i,p){<J*)~Htk,Xk)AB'''] I . 
From the construction of Pfc+i and the fact that 4>'^ is convex it holds with (27) 

E [0"(tfc +T,Xfe+l,Pfc+l)] > E [<^^{tk+T,Xk+l,Pk) + (^0"(tfc +T,Xfc+l,Pfc),Pfe+l -pfc) 

= E[r{ik+T,Xk+uPk)] 



(40) 
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□ 



It remains to get a suitable estimate for Zk{xk,Pk+i)- Since (j)'^ is uniformly Lipscliitz continuous in 
X, it holds by Taylor expansion in x 

Sk{xk,Pk+i) = -E[(j)^tk+r,Xk+up){<7*)~\tk,Xk)AB''']\ 

'Y I/'— Pfc + i 

= [4,-'{h+T,Xk,p){<J*)-\h,Xk)l^B'-] I 

+ [i?,0^(tfc+T,Xfc,p)|AS''=|'] \^+0{t) 
= ^-E[D.,<P^{h + T,xu,p)\AB'''?] I^^p^^^ +0(r). 

Furthermore since Dx4>^ is Lipschitz continuous in p it holds with (38) 

E [\D,(f>^{tk+T,Xk,Pk+i)\AB'''\^ - a,(j)^tk+T,Xk,Pk)\AB'''\^\] 
<cE[|pfc+i -pkWAB'"]^] <cTi 

So from (40) we have 

> E (j)^{tk+T,Xk+i,Pk) ~ (l>^{tk,Xk,Pk) + TH{ik,XkjDr,(f{^^ 

-c{Ti +tO{t)) 
which implies (25) since (jf is equal to cj) up to a linear shift. 

5 Concluding Remarks and Outlook 

In this paper we gave an approximation scheme for the value function of a stochastic differential game 
with incomplete information. It is natural to ask whether this approximation might be used to deter- 
mine optimal feedback strategies for the informed player. In the deterministic games with complete 
information it is well known that the answer is positive (see the step by step motions associated with 
feedbacks in [5D] ) . The case of deterministic games with incomplete information has been treated in 

The approximation of optimal strategies for stochastic differential games is a more delicate topic even 
in the case with complete information. [2] - also considering the game under a Girsanov transform 
- gives a partwise answer under a weak Lipschitz assumption of the feedback control. The result is 
shown by using approximations of BSDEs however not in a completely discrete framework. In the 
very recent paper |16| approximately Markov strategies are constructed with an approximation that 
in contrast to ours takes into account the actions of the other player during the time intervals. This 
however makes the approximation much harder to implement. 

In fact, if we use the approximation for the construction of optimal strategies for the informed player 
we are in the same situation as 25 . For the approximation of the value function in [5T] nearly op- 
timal policies are constructed which possess a certain optimality in the approximative discrete time 
games instead of the continuous time one. To the authors knowledge the problem of finding an effi- 
cient approximation of optimal strategies in stochastic differential games (with or without incomplete 
information) is open and poses an interesting problem for further research. 
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